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j| Managers today must make decisions in the face of 

f ' considerable uncertainty. Not only are data missing dr 

!; incomplete, in many circumstances there are no well-; 

defined procedures' for. combining the existing data 
; ; objectively to produce a unique solution. To help 

tJV.. '".managers in such circumstances, attempts have been made 
'( * ' : to develop' methods that, while not entirely objective, do 

| . minimize the subjective element in the decision-making 

| . : process. The Delphi technique is one such method. 

3 ; , In situations such as planning or forecasting where 

i| there is a particularly significant amount of uncertainty, 

informed judgment often seems more important or more 
easily obtainable than quantitative data. A common 
practice, therefore, is to seek such judgments from 
experts who are presumed to have knowledge of whatever 
data is available, thereby, theoretically, combining both 
objective and subjective bases for decisions. A group of 
experts rather than a single source is usually consulted 
whenever possible — the principle that "two heads are 
better than one." The Delphi technique was developed 
in an attempt to make the most effective use of informed 
intuitive judgment by creating conditions under which a 
group of experts can perform most ably and their answers 
can be combined into a single group opinion. 

Technique 

The term "Delphi technique" implies more than 
simply "consensus of opinion." Three features distinguish 
the Delphi technique from a simple round-table or panel 
technique: 

1 . Anonymous response, 

2. Iteration and controlled feedback, and 

3. Statistical derivation of a group response. 

During a Delphi procedure, participants normally do 

not meet face to face but are questioned individually and 
never told which other participants gave which responses. 
The anonymous response, which is basic to Delphi, is 
intended to avoid a number of the problems inherent in 
face-to-face discussion: 

1. A group may be persuaded by individuals who 
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dominate by virtue of rank, reputation, or (sheer) 
personality. - u 

2. A member ‘of a .group may fear "losing face" by 
abandoning a publicly expressed opinion. 

3- The "bandwagon" effect often leads to adoption 
of solutions that are less than optimal. 

4‘. Group discussions generate large amounts of 
"noise," irrelevant discussion and sidetracks that 
contribute nothing to the central issue! 

5. It is difficult to arrange for a number of experts 
to get together at one place and time, 

Experiments at RAND, mostly on forecasting future 
trends in specific technical areas and on answering factual 
technical questions, indicate that anonymous responses do 
produce better results than face-to-face group discussion. 

There is no doubt, however, that intellectual 
interaction can be valuable in forming judgment, and it is 
to provide this interaction in the absence of personal 
contact that the second feature of the Delphi technique is 
included. Responses to the first set of questions are 
tabulated and sent to the participants, who are asked to 
respond again, using the information provided by the 
other responses. This feed back -and -response cycle may 
continue through several iterations. Sometimes 
respondents are asked to give reasons or are allowed to 
suggest further questions or ask for data. Thus, new ideas 
can be disseminated for consideration by the whole group, 
and also respondents can get a feel for where the majority 
opinion lies. There is a danger, however, that the persons 
tabulating responses and coordinating the Delphi exercise 
can consciously or accidentally distort the feedback and 
bias the results. 

The third feature of Delphi is production of a single 
group response rather than simply a tabulation of 
individual responses. The individual responses are 
aggregated in a manner that assures that every opinion 
contributes to the final group opinion. Exactly how to 
combine the opinions in itself may be a difficult problem, 
especially when they cannot be expressed numerically. 

One modification to the basic Delphi technique is a 
ranking of experts and weighing of their opinions. 
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Unfortunately, ranking the experts may be as difficult as 
making the original decision, and attempts to use self- 
ratings by each expert may measure ego as much as 
expertise.. 

Critique 

In the absence of facts, certainly it is normally better to 
trust the judgment of experts than non-experts. Both 
statistical theory and experiment have shown that one is 
more likely to be close to the correct or best answer by 
sampling, many responses than by depending on just one. 
However, the choice of experts is crucial, in both the area 
of choosing general classes, and. the area of naming specific 
names. The most famous uses of Delphi have been in the 
area of technical forecasting. For example, financial 
experts might, be asked to predict market conditions five 
years in the future, or technological experts asked by what 
year they would expect a specific technological advance in 
. their field to have been .made. In such- cases, the choice of 
experts is fairly obvious. 

Not all fields, however, have the right types of experts 
to answer certain kinds of questions. The intelligence 
community, for example, presents two different types of 
stumbling blocks to the selection of experts capable of 
giving judgments on important issues. First, a basic 
hypothesis of the Delphi method is that the experts carry 
a technical data base in their heads to back up their 
judgments. , It is not normal Delphic procedure for the 
persons conducting the questionnaire process to supply 
data along with the questions. The structure of the 
community, however, is such that the persons with the 
requisite technical knowledge are usually hot on a level to 
make judgments, while those at the appropriate levejs 
often must depend on others for data. Second, the 
intelligence community is composed of specialists and 
frequently does not possess people capable of judging all 
facets of a complex problem. For example, in a survey 
designed to assess the value of Flint sources, a Navy 
expert might be able to reply to the value of a specific 
source of the Navy, but he cannot assess its value to the 
Air Force, and therefore he cannot assess its value to the 
community. 

A second weakness of Delphi is its tendency to ignore 
facts. Delphi is intended for situations where the available 
information by itself is insufficient for decision-making, 
but it is still desirable to include whatever data are avail- 
able in the decision process. Delphi does not guarantee 
that the judges employ any factual data at all in forming 
their opinions, nor does it encourage them to seek new 
data or use any existing data base. An experiment at 
RAND showed that supplying relevant facts, to the ex- 
perts along with questions did, indeed, improve answers. 
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The selection of the facts in itself, however, may intro- 
duce bias; and presenting facts with the questions may 
predispose busy experts against doing any research 
themselves. Use of data should be maximized in a good 
. decision-making technique.. 

Another danger presented by Delphi's comparative 
neglect of facts is the likelihood that widely held 
misconceptions will be continued. Scientific progress is 
based on the fact that a hard look at data can often reveal 
long-cherished ideas to be wrong, but if decisions depend 
solely on judgment, such recognition of error is unlikely. 
Many managerial decisions involve value judgments, 
which are particularly likely to be affected by prejudices, 
even among professionals, or, by organizational 
considerations that make., unbiased judgment extremely 
difficult. . 

Finally, major intelligence community problems 
involve' a. very complex system. Delphi tends to approach 
such problems in an oversimplified.manner by isolating, 
.specifics from the system and lookirig at them as. 
individual and unrelated bits. The net result is that the 
opinions on these isolated pieces omit the serious 
interrelationship of these; pieces. The technical way of 
stating this is that the systems approach is lost or 
overlooked. 

In sum, it is erroneous to expect the Delphi technique 
to be applicable to all situations or to believe that the 
results are objectively derived. Development of truly 
scientific aids to managerial decision making must still be 
pursued. If expert advice is the only available source of 
information, the Delphi technique does represent a way to 
elicit and combine the opinions. It must be remembered, 
however, that the individual responses are still subjective 
judgments ; and; that statistically aggregating subjective 
judgments does not make the results objective. 

In a field as important and complex as the production 
of intelligence, it is important to be aware of the validity 
of sources and to take care to use the best sources possible. 
The manager to whom the Delphi technique is 
recommended as a scientific management tool should 
realize its basically subjective nature. 
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